Microscope device and image acquisition method

ABSTRACT

A microscope apparatus ( 1 A) includes a biological sample table ( 11 ) that supports the biological sample (B), an objective lens ( 12 ) disposed to face the biological sample table ( 11 ), a laser light source ( 13 ) that outputs light with which the biological sample (B) is irradiated via the objective lens ( 12 ), a shape measurement unit ( 20 ) that acquires a surface shape of the biological sample (B), a control unit ( 40 ) that generates aberration correction hologram data for correcting an aberration caused by the surface shape of the biological sample (B) on the basis of information acquired in the shape measurement unit ( 20 ), a first spatial light modulator ( 33 ) to which a hologram based on the aberration correction hologram data is presented and that modulates the light output from the laser light source ( 31 ), and a photodetector ( 37 ) that detects an intensity of light to be detected (L 2 ) generated in the biological sample (B). Thus, a microscope apparatus and an image acquisition method capable of suppressing a decrease in condensing intensity of irradiation light inside a biological sample and spreading of a condensing shape are realized.

TECHNICAL FIELD

One aspect of the present invention relates to a microscope apparatusand an image acquisition method.

BACKGROUND ART

Patent Literature 1 describes a technology regarding a microscopecapable of performing aberration correction. In this microscope, asample is irradiated with excitation light, fluorescence generated inthe sample is split and condensed, and a condensing image is captured bya camera. A spot of the condensing image is a diffraction limit imagewhen there is no aberration, and has a shape with a distortion whenthere is an aberration. Therefore, in this microscope, wavefrontaberration correction is applied to attempt to improve a spot shape.Accordingly, a plurality of aberration conditions corresponding to aplurality of spot shapes are stored, selected, and applied to achievehigh speed.

CITATION LIST Patent Literature

[Patent Literature 1] Japanese Unexamined Patent Publication No.2013-160893

SUMMARY OF INVENTION Technical Problem

A characteristic of the spatial light modulator is that variousaberrations can be corrected by controlling a wavefront. For example, ina laser scanning microscope, there is a problem in that a sphericalaberration is generated due to mismatch in refractive index between anobservation target and a surrounding medium (for example, air, water,silicone oil, or oil), and a condensing point of the irradiation lightextends in an optical axis direction inside the observation target, butthe extension of the condensing point inside the observation target canbe suppressed by controlling the wavefront of the irradiation lightusing a spatial light modulator and correcting the spherical aberration.Since a fine image can be obtained by such aberration correction, such amicroscope is preferably used, for example, to obtain an internalobservation image of a biological sample, such as a blood vessel imageof a brain.

In many cases, however, a surface of the biological sample is not flat.Accordingly, an aberration caused by a surface shape of the biologicalsample occurs, in addition to the above-described aberration. Forexample, when a fish such as a zebrafish is observed as a biologicalsample, a surface shape of the fish can be regarded as substantially acylindrical shape, and an astigmatism aberration occurs due to thecylindrical shape. Further, for example, a biological sample such as acell sample includes a blood vessel, a cell tissue or the like.Accordingly, there are substances with different refractive indexes suchas red blood cells or lipids. Therefore, an aberration occurs inside thebiological sample. When irradiation light is under an influence ofaberration, there is a problem in that a condensing intensity of theirradiation light is weakened inside the biological sample or acondensing shape spreads.

One aspect of the present invention has been made in view of suchproblems, and an object thereof is to provide a microscope apparatus andan image acquisition method capable of suppressing a decrease incondensing intensity of irradiation light inside a biological sample andspreading of a condensing shape.

Solution to Problem

To solve the problems described above, a microscope apparatus accordingto one aspect of the present invention is an apparatus for acquiring animage of a biological sample, and includes a biological sample table forsupporting the biological sample; an objective lens disposed to face thebiological sample table; a light source for outputting light with whichthe biological sample is irradiated via the objective lens; a shapeacquisition unit for acquiring information on at least one of a surfaceshape of the biological sample and a structure directly under thesurface of the biological sample; a hologram generation unit forgenerating aberration correction hologram data for correcting anaberration caused by the at least one on the basis of the informationacquired by the shape acquisition unit; a spatial light modulator towhich a hologram based on the aberration correction hologram data ispresented and for modulating the light output from the light source; aphotodetector for detecting an intensity of light generated in thebiological sample and outputs a detection signal; and an imagegeneration unit for generating an image of the biological sample on thebasis of the detection signal.

Further, an image acquiring method according to one aspect of thepresent invention is a method for acquiring an image of a biologicalsample and includes a step of acquiring information on at least one of asurface shape of the biological sample supported by a biological sampletable facing an objective lens and a structure directly under thesurface of the biological sample (shape acquisition step); a step ofgenerating aberration correction hologram data for correcting anaberration caused by the at least one on the basis of the informationacquired in the shape acquisition step (hologram generation step); astep of presenting a hologram based on the aberration correctionhologram data to a spatial light modulator, modulating the light outputfrom a light source using the spatial light modulator, and irradiatingthe biological sample with light after modulation (light irradiationstep); a step of detecting an intensity of light generated in thebiological sample and outputting a detection signal (light detectionstep); and a step of generating an image of the biological sample on thebasis of the detection signal (image generation step).

In the microscope apparatus and the image acquisition method,information on at least one of the surface shape of the biologicalsample and the structure directly under the surface of the biologicalsample is acquired. The aberration correction hologram data forcorrecting an aberration is generated on the basis of that information.Further, the irradiation light is modulated by the hologram based on thedata. Accordingly, since an aberration caused by at least one of thesurface shape of the biological sample and the structure directly underthe surface of the biological sample is preferably corrected, it ispossible to suppress a decrease in condensing intensity of theirradiation light inside the biological sample and spreading of thecondensing shape.

Advantageous Effects of Invention

According to the microscope apparatus and the image acquisition methodaccording to an aspect of the present invention, it is possible tosuppress a decrease in condensing intensity of the irradiation lightinside the biological sample and spreading of a condensing shape.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating a configuration of a microscopeapparatus according to a first embodiment.

FIG. 2 is a flowchart illustrating an operation of a microscopeapparatus.

FIG. 3 is a diagram schematically illustrating a state of generation ofan aberration.

FIG. 4 conceptually illustrates a state of irradiation light when aboundary between a biological sample and outside thereof is inclinedfrom a plane perpendicular to an optical axis in a case in whichirradiation light that is plane waves not subjected to aberrationcorrection is condensed by an objective lens.

FIG. 5 is a diagram illustrating a method of calculating a wavefront forconcentrating rays on any point on an optical axis.

FIG. 6 schematically illustrates a case in which there is a boundarybetween two refractive indexes.

FIGS. 7(a) and 7(b) are diagrams illustrating a wavefront obtained usingbackpropagation analysis, in which a phase is indicated by shading.

FIGS. 8(a) to 8(d) are diagrams when a biological sample is viewed fromthe side, and conceptually illustrate a state in which irradiation lightis condensed while passing through an internal structure including mediawith different refractive indexes.

FIG. 9 is a diagram illustrating a configuration of a microscopeapparatus of a second embodiment.

FIG. 10 is a flowchart illustrating an operation of a microscopeapparatus and an image acquisition method according to this embodiment.

FIG. 11 is a diagram illustrating an operation of a first modificationand is a diagram when a surface of a biological sample is viewed in anoptical axis direction.

FIGS. 12(a) and 12(b) are diagrams illustrating a state of scanning ofthe first modification example.

FIG. 13 is a diagram illustrating a configuration of a microscopeapparatus according to a second modification example.

FIG. 14 is a diagram illustrating a configuration of a microscopeapparatus according to a fourth modification example.

FIG. 15 is a diagram illustrating a configuration of a microscope unitand a shape measurement unit of a microscope apparatus according to afifth modification.

FIG. 16 is a diagram illustrating a configuration of a microscopeapparatus according to a sixth modification example.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of a microscope apparatus and an imageacquisition method according to an aspect of the present invention willbe described with reference to the accompanying drawings. In thedescription of the drawings, the same elements are denoted with the samereference numerals, and repeated description will be omitted.

First Embodiment

FIG. 1 is a diagram illustrating a configuration of a microscopeapparatus 1A according to an embodiment of one aspect of the presentinvention. The microscope apparatus 1A is an apparatus for acquiring animage of a biological sample B. The microscope apparatus 1A includes amicroscope unit 10, a shape measurement unit 20, an image acquisitionunit 30, and a control unit 40, as illustrated in FIG. 1.

The microscope unit 10 irradiates the biological sample B withirradiation light L1 from a shape measurement unit 20 and an imageacquisition unit 30, which will be described below, and outputs light tobe detected L2 from the biological sample B to each of the shapemeasurement unit 20 and the image acquisition unit 30. The light to bedetected L2 is reflected light of the irradiation light L1, harmonics ofthe irradiation light L1, or fluorescence excited by the irradiationlight L1. The microscope unit 10 includes a biological sample table 11,an objective lens 12, an objective lens moving mechanism 13, and a beamsplitter 14.

The biological sample table 11 is a plate-like member for supporting thebiological sample B (or a vessel that accommodates the biological sampleB). The biological sample table 11 is formed of, for example, a materialthat transmits the irradiation light L1 and the light to be detected L2,such as glass or plastic. The biological sample table 11 is, forexample, a glass slide, a bottomed dish, or a microplate. In thisembodiment, the irradiation light L1 is radiated to a back surface ofthe biological sample table 11, transmitted through the biologicalsample table 11, and radiated to the biological sample B. Further, thelight to be detected L2 from the biological sample B is transmittedthrough the biological sample table 11 and output from the back surfaceof the biological sample table 11.

The objective lens 12 is disposed to face the biological sample table11, and condenses the irradiation light L1 on the inside of thebiological sample B. Further, the objective lens 12 collects the lightto be detected L2 from the biological sample B.

Although a common objective lens is used for the objective lens for theirradiation light L1 and the objective lens for the light to be detectedL2 in this embodiment, the objective lens for the irradiation light L1and the objective lens for the light to be detected L2 may beindividually provided. For example, an objective lens having a highnumerical aperture (NA) may be used for the irradiation light L1 andlocally focused by aberration correction. Further, an objective lenshaving a large pupil may be used for the light to be detected L2 toextract a larger amount of light. The objective lens for the irradiationlight L1 and the objective lens for the light to be detected L2 may bearranged so that the biological sample B is interposed therebetween andtransmitted light in the biological sample B of the irradiation light L1may be acquired as the light to be detected L2.

The objective lens moving mechanism 13 is a mechanism for moving theobjective lens 12 in the optical axis direction of the irradiation lightL1. The objective lens moving mechanism 13 includes a stepping motor, apiezo-actuator, or the like.

The beam splitter 14 divides and combines an optical path between themicroscope unit 10 and the image acquisition unit 30 and an optical pathbetween the microscope unit 10 and the shape measurement unit 20.Specifically, the beam splitter 14 reflects the irradiation light L1reaching the microscope unit 10 from the image acquisition unit 30,toward the objective lens 12. Further, the beam splitter 14 reflects thelight to be detected L2 collected by the objective lens 12, toward theimage acquisition unit 30. On the other hand, the beam splitter 14transmits light L32 from the shape measurement unit 20 and reflectedlight of the light L32 in the biological sample B. The beam splitter 14may include, for example, a half mirror or a dichroic mirror. Themicroscope unit 10 further includes a reflective mirror 15 that changesan optical axis direction of the light L32.

The shape measurement unit 20 is a shape acquisition unit in thisembodiment. The shape measurement unit 20 includes a detector 24, and isoptically coupled to the objective lens 12. The shape measurement unit20 acquires information on a shape with a refractive index distributionof the biological sample B, that is, information on a surface shape ofthe biological sample B (a shape according to a refractive indexdifference due to the biological sample B and the surrounding (air)).The shape measurement unit 20 may be, for example, an interference lightmeasurement unit that measures the surface shape of the biologicalsample B using a Michelson interferometer. In this case, the shapemeasurement unit 20 includes a coherent light source 21, a beam splitter22, a reference light mirror 23, and a detector 24, as illustrated inFIG. 1.

The coherent light source 21 generates coherent light L3 with which thebiological sample B is irradiated. The coherent light source 21preferably includes, for example, a semiconductor laser element.

The beam splitter 22 causes the coherent light L3 from the coherentlight source 21 to separate into reference light L31 and light L32directed to the microscope unit 10. Further, the beam splitter 22reflects the reference light L31 reflected by the reference light mirror23 and transmits reflected light from the surface of the biologicalsample B of the light L32. Accordingly, the beam splitter 22 combinesthe lights to generate interference light L4. The interference light L4is input to the detector 24. The reference light mirror 23 may beconfigured to be movable with respect to the optical axis direction ofthe reference light L31 or may be fixed.

The detector 24 detects the interference light L4 combined by the beamsplitter 22 and outputs a detection signal S1. The detector 24 includesa two-dimensional photodetector such as a CCD image sensor or a CMOSimage sensor.

The shape measurement unit is not limited to the configuration of thisembodiment. For example, the shape measurement unit may use aninterference measurement scheme such as a Mirau type or a Linnik type.Alternatively, the shape measurement unit may include a confocalreflectance microscope or may include a common path interferometer.According to such a microscope, it is possible to preferably measure thesurface shape of the biological sample B using focusing information.Further, the shape measurement unit may use evanescent light.Accordingly, shape recognition such as recognition as to whether or notthe biological sample B is grounded on the biological sample table 11 isfacilitated.

The image acquisition unit 30 detects the light to be detected L2 fromthe biological sample B to generate an image. Hereinafter, an example ofa fluorescence optical system in a case in which the light to bedetected L2 is fluorescence from the biological sample B will bedescribed, but the irradiated light generation unit is not limitedthereto, and a reflective optical system in a case in which the light tobe detected L2 is reflected light from the biological sample B or atransmissive optical system in a case in which the light to be detectedL2 is transmitted light from the biological sample B may be used. Theimage acquisition unit 30 of this embodiment includes a laser lightsource 31, a beam expander 32, a first spatial light modulator (SLM) 33,a dichroic mirror 34, an optical scanner 35, a second spatial lightmodulator (SLM) 36, a detector 37, and a filter 38.

The laser light source 31 is a light irradiation unit in thisembodiment, and irradiates the biological sample B with light L5 via theobjective lens 12. The laser light source 31 is a light source thatoutputs the light L5 with which the biological sample B is irradiatedvia the objective lens 12. The laser light source 31 is opticallycoupled to the objective lens 12. The light L5 is, for example, laserlight including an excitation wavelength of the biological sample B. Thelaser light source 31 includes, for example, a semiconductor laserelement. The beam expander 32 includes, for example, a plurality oflenses 32 a and 32 b arranged side by side on an optical axis of thelight L5, and adjusts a size of a cross-section perpendicular to theoptical axis of the light L5.

A first spatial light modulator 33 is optically coupled to the laserlight source 31. The first spatial light modulator 33 presents ahologram including an aberration correction hologram for correcting anaberration caused by the surface shape of the biological sample B. Thefirst spatial light modulator 33 modulates the light L5 from the laserlight source 31 to generate the irradiation light L1 with which thebiological sample B is irradiated. The surface shape of the biologicalsample B is measured by the shape measurement unit 20 described above.The first spatial light modulator 33 may be of a phase modulation typeor of an amplitude (intensity) modulation type. Further, the firstspatial light modulator 33 may be either of a reflection type or of atransmission type. The aberration correction hologram will be describedin detail below.

The dichroic mirror 34 transmits one of the irradiation light L1 fromthe first spatial light modulator 33 and the light to be detected L2from the microscope unit 10 and reflects the other. In the exampleillustrated in FIG. 1, the dichroic mirror 34 transmits the irradiationlight L1 and reflects the light to be detected L2.

The optical scanner 35 scans an irradiation position of the irradiationlight L1 in the biological sample B by moving an optical axis of theirradiation light L1 within a plane perpendicular to the optical axis ofthe irradiation light L1. The optical scanner 35 includes, for example,a galvanometer mirror, a resonant mirror, a MEMS mirror, or a polygonmirror. Further, the light to be detected L2 from the biological sampleB is detected through the optical scanner 35. Thus, it is possible tomatch the optical axis of the irradiation light L1 with the optical axisof the light to be detected L2.

The second spatial light modulator 36 is optically coupled to theobjective lens 12 and the detector 37. The second spatial lightmodulator 36 presents an aberration correction hologram for correctingaberration caused by the surface shape of the biological sample B. Thesecond spatial light modulator 36 modulates the light to be detected L2from the dichroic mirror 34. The surface shape of the biological sampleB is measured by the shape measurement unit 20 described above. Thesecond spatial light modulator 36 may be of a phase modulation type ormay be of an amplitude (intensity) modulation type. Further, the secondspatial light modulator 36 may be of either a reflection type or atransmission type.

Further, when a pinhole is disposed at a preceding stage from thedetector 37, it is preferable for a hologram for condensing the light tobe detected L2 on the pinhole to be presented to the second spatiallight modulator 36, in addition to the aberration correction hologram.Thus, it is possible to obtain a confocal effect. When the light to bedetected L2 such as fluorescence generated from the biological sample Bis detected using a multi-photon absorption effect such as two-photonabsorption, a hologram for condensing the light to be detected L2 on thedetector 37 is combined with the aberration correction hologram andpresented to the second spatial light modulator 36, such that theconfocal effect can be obtained. The aberration correction hologram willbe described in detail below.

The detector 37 is an light detection unit of this embodiment. Thedetector 37 is optically coupled to the objective lens 12. The detector37 detects a light intensity of the light to be detected L2 that isoutput via the objective lens 12 from the biological sample B, andoutputs a detection signal S2. The detector 37 may be a point sensorsuch as a photomultiplier tube (PMT), a photodiode, or an avalanchephotodiode. Alternatively, the detector 37 may be an area image sensorsuch as a CCD image sensor, a CMOS image sensor, a multi-anode PMT, or aphotodiode array. A condensing lens 37 a may be disposed directly beforethe detector 37.

The filter 38 is disposed on an optical axis between the dichroic mirror34 and the detector 37. The filter 38 cuts a wavelength of theirradiation light L1 and a wavelength of fluorescence or the likeunnecessary for observation from the light that is input to the detector37. The filter 38 may be disposed at a preceding stage or a subsequentstage from the condensing lens 37 a.

The image acquisition unit 30 of this embodiment further includes amirror 39 a and a reflective member 39 b, in addition to the aboveconfiguration. The mirror 39 a bends the optical axis of the irradiationlight L1 and the light to be detected L2 in order to optically couplethe optical scanner 35 to the beam splitter 14 of the microscope unit10. The reflective member 39 b is a prism having two reflective surfacesand is disposed to face the second spatial light modulator 36. Thereflective member 39 b reflects the light to be detected L2 from thedichroic mirror 34 toward the second spatial light modulator 36 at oneof the reflective surfaces, and reflects the light to be detected L2from the second spatial light modulator 36 toward the detector 37 at theother reflective surface.

When a distance between the objective lens 12 and the first spatiallight modulator 33 is long, at least one 4f optical system may beprovided on an optical axis of the irradiation light L1 and the light tobe detected L2. As an example, two 4f optical systems 51 and 52 areillustrated in FIG. 1. The 4f optical systems 51 and 52 serve totransfer a wavefront of the irradiation light L1 generated in the firstspatial light modulator 33 to a rear-side focus point of the objectivelens 12. The 4f optical system may include a telecentric relay opticalsystem. Further, when the objective lens 12 is very close to the firstspatial light modulator 33, the 4f optical system may be omitted.

The control unit 40 includes a processor. The control unit 40 controlsthe microscope unit 10, the shape measurement unit 20, and the imageacquisition unit 30. For example, the control unit 40 controls aposition in the optical axis direction of the objective lens 12 usingthe objective lens moving mechanism 13 in the microscope unit 10.Further, the control unit 40 moves the biological sample table 11 thatsupports the biological sample B in a direction crossing the opticalaxis direction. Further, the control unit 40 performs control of thecoherent light source 21, the detector 24, and the reference lightmirror 23 of the shape measurement unit 20. Further, the control unit 40controls the laser light source 31, the first spatial light modulator33, the optical scanner 35, the second spatial light modulator 36, andthe detector 37 of the image acquisition unit 30. The control unit 40 ofthis embodiment includes an input device 41 such as a mouse or akeyboard, a display device (display) 42, and a computer 43.

Further, the control unit 40 constitutes a portion of the shapeacquisition unit in this embodiment. The control unit 40 is electricallycoupled to the detector 24 of the shape measurement unit 20. The controlunit 40 receives the detection signal S1 that is output from thedetector 24 of the shape measurement unit 20. The control unit 40acquires information on the surface shape of the biological sample Busing a method using a Fourier transform or a λ/4 phase-shiftinginterferometry on the basis of the detection signal S1. Further, thecontrol unit 40 is a hologram generation unit in the embodiment. Thecontrol unit 40 generates aberration correction hologram data forcorrecting aberration caused by the surface shape of the biologicalsample B on the basis of the obtained information. The aberrationcorrection hologram data is provided to the first spatial lightmodulator 33 and the second spatial light modulator 36.

Further, the control unit 40 is an image generation unit in thisembodiment. The control unit 40 generates an image regarding thebiological sample B on the basis of the detection signal S2 from thedetector 37 and information, on an irradiation position of the opticalscanner 35. The generated image is displayed on the display device 42.The control unit 40 may realize a function of controlling the respectiveunits, a function as the shape acquisition unit, a function as thehologram generation unit, and a function as an image generation unitusing the same processor or may realize the functions using differentprocessors.

FIG. 2 is a flowchart illustrating an operation of the microscopeapparatus 1A described above. An image acquisition method according tothis embodiment will be described with reference to FIG. 2.

First, the biological sample B is placed on the biological sample table11. Then, the light L3 is output from the light source 21 of the shapemeasurement unit 20, and the detector 24 detects the interference lightL4 of the reflected light from the surface of the biological sample Band the reference light L31. Thus, an interference pattern is observedon the surface of the biological sample B. In the control unit 40,information on the surface shape of the biological sample B is acquiredon the basis of the interference pattern (shape acquisition step S11).

Subsequently, aberration correction hologram data for correcting theaberration caused by the surface shape of the biological sample B isgenerated by the control unit 40 on the basis of the informationacquired in the shape acquisition step S11 (hologram generation stepS12). Subsequently, the hologram based on the aberration correctionhologram data is presented to the first spatial light modulator 33 andthe second spatial light modulator 36. The light L5 emitted from thelaser light source 31 is modulated by the first spatial light modulator33, and the biological sample B is irradiated with the irradiation lightL1 after modulation (light irradiation step S13).

Subsequently, the detector 37 detects the intensity of the light to bedetected L2 generated in the biological sample B (light detection stepS14). In this case, the light to be detected L2 is modulated by thesecond spatial light modulator 36 and then input to the detector 37. Inthis embodiment, the light irradiation step S13 and the light detectionstep S14 are repeatedly performed (or simultaneously continuously) whilescanning the irradiation light L1 using the optical scanner 35. Then,the image of the biological sample B is generated on the basis of thedetected information in the light detection step S14 in the control unit40 (image generation step S15).

Effects obtained by the microscope apparatus 1A and the imageacquisition method of this embodiment including the above configurationwill be described.

In many cases, the surface of the biological sample B is not flat.Therefore, an aberration caused by the surface shape of the biologicalsample B occurs. For example, if a surface shape of a fish is assumed tobe substantially cylindrical in shape when a fish such as a zebrafish isobserved as the biological sample B, a strong astigmatism aberrationoccurs when the irradiation light L1 in a condensing process passesthrough the surface. When a numerical aperture (NA) of the objectivelens 12 is small or observation is performed at a shallow position inthe biological sample B, an influence thereof is less, but when thenumerical aperture (NA) is great or the observation is performed at adeep position, the influence cannot be neglected. There is a problem inthat a condensing intensity of the irradiation light L1 inside thebiological sample B becomes weak or a condensing shape spreads when theirradiation light L1 is under an influence of such an aberration.Accordingly, for example, degradation of resolution due to extension ofan excitation region, a decrease in fluorescence intensity, and adecrease in a signal to noise ratio (SN ratio) due to an increase inbackground noise occur.

FIG. 3 is a diagram schematically illustrating a state of occurrence ofan aberration. In FIG. 3, a curve B1 represents a boundary between thesurface of the biological sample B, that is, the biological sample B andoutside thereof. It is assumed that a refractive index outside of thebiological sample B is n1 and a refractive index of the inside of thebiological sample B is n2 (≠n1). The irradiation light L1 is condensedon the inside (directly under the surface) of the biological sample B bythe objective lens 12. In this case, light L11 that passes through thevicinity of an optical axis A2 of the objective lens 12 in theirradiation light L1 goes straight toward the condensing point C underalmost no influence from the surface shape of the biological sample B.On the other hand, light L12 passing through a position apart from theoptical axis A2 of the objective lens 12 in the irradiation light L1 isrefracted under the influence of the surface shape of the biologicalsample B and deviates from the condensing point C. Due to thisphenomenon, a condensing intensity of the irradiation light L1 insidethe biological sample B becomes weak and a condensing image D spreads.

In the microscope apparatus 1A and the image acquisition method of thisembodiment, information on the surface shape of the biological sample Bis acquired. On the basis of the information, aberration correctionhologram data for correcting an aberration is generated. Further, theirradiation light L1 is modulated by a hologram based on the data. Thus,since the aberration caused by the surface shape of the biologicalsample B is preferably corrected, it is possible to suppress a decreasein the condensing intensity of the irradiation light L1 inside thebiological sample B and spreading of the condensing image D.

Further, as in this embodiment, the hologram based on the aberrationcorrection hologram data may be presented to the second spatial lightmodulator 36, and the light to be detected L2 generated in thebiological sample B may be modulated by the second spatial lightmodulator 36. Thus, the influence of the aberration caused by thesurface shape of the biological sample B on the light to be detected L2is corrected, and accordingly, when confocal using a pinhole isperformed, the light to be detected L2 passing through the pinholeincreases such that a clearer image of the biological sample B can beobtained.

Here, a method of designing the aberration correction hologram will bedescribed in detail. FIG. 4 conceptually illustrates a state ofirradiation light L6 when a boundary B1 between the biological sample Band outside thereof is inclined by an angle a from a plane Hperpendicular to the optical axis A2 in a case in which the irradiationlight L6 that is plane waves not subjected to the aberration correctionis condensed by the objective lens 12. Here, lights L6 a and L6 b inFIG. 4 are rays passing through the vicinity of a center of theobjective lens 12 and are referred to as paraxial rays. Further, lightsL6 c and L6 d in FIG. 4 are rays passing through the vicinity of an edgeof the objective lens 12, and are referred to as outer peripheral rays.

When the rays pass through a boundary between the biological sample Band outside thereof, a relationship between an incidence angle θ₁ and anoutput angle θ₂ of the rays with respect to the boundary B1 is obtainedusing Snell's law expressed by the following Equation (1).

[Math. 1]

sin(θ₁±α)=n ₂ sin(θ₂±α)   (1)

Since the incidence angle is small for the paraxial rays, the amount ofchange in the incidence angle and the output angle is small. On theother hand, since the incidence angle is large for the outer peripheralrays, the amount of change in the incidence angle and the output angleis large. Further, since the boundary B1 is inclined with respect to aplane perpendicular to the optical axis A2, the paraxial rays and theouter peripheral rays do not overlap on the optical axis. Accordingly,various aberrations occur and, as a result, the condensing image becomesa distorted image that is different from a diffraction limit image.

FIG. 5 is a diagram illustrating a method of calculating a wavefront sothat rays are concentrated on any point O′ on the optical axis A2. Anarc Q, Q′ is a wavefront after the plane waves pass through theobjective lens 12 with a focal length f and, for example, is a sphericalcrown with a certain radius. If there is no biological sample B, theirradiation light L6A is condensed on another point O on the opticalaxis A2, as indicated by a two-dot chain line in FIG. 5. Here, an angleθ_(max) between a line segment OQ (or OQ′) and a line segment OT isexpressed by the following Equation (2). Here, NA is a numericalaperture of the objective lens 12. Further, T is an intersection of thearc Q, Q′ and the optical axis A2.

[Math. 2]

θ_(max)=sin⁻¹(NA/n1)   (2)

The rays of the irradiation light L6A are input to the objective lens 12from the left on the paper surface, and proceed to the right of thepaper surface as convergent light. This direction is defined as apositive propagation direction. For example, an optical path from thepoint O′ to a point W on an arc QQ′ via a point V on the boundary B1 isobtained through backpropagation. In this embodiment, for example, theoptical path is obtained through backpropagation using a method such asbackward ray tracing, wavefront propagation, and electromagnetic fieldanalysis to be described below.

<Backward Ray Tracing>

In this embodiment, in a previous step of optical path calculation, astructure of the biological sample B is recognized using interferencemeasurement. Therefore, a refractive index distribution in thebiological sample B is estimated from the structure of the biologicalsample B, and the boundary B1 of the refractive index is obtained. Theboundary B1 of this refractive index is subjected to polynomialapproximation or map conversion so that the position can be specified.

Then, a path along which rays reach a point W on the arc QQ′ from thepoint O′ via the point V on the refractive index boundary B1′ and anoptical path length are determined. If the rays from the point O′propagate back, the rays reach the point V on the refractive indexboundary B1. The point V is obtained by the above-described polynomialapproximation or the like. The rays are refracted at the point V usingSnell's law due to a refractive index difference before and after theboundary B1. In this embodiment, since the boundary B1 that is notuniform (that has irregular unevenness) is assumed, a three-dimensionalSnell's law using a vector and a cross product as shown in Equation (3)below is applied.

[Math. 3]

n ₁ m×VW=n ₂ ·m×O′V   (3)

Here, × in Equation (3) denotes a cross product. Further, in Equation(3), m is a normal vector at the point V, and VW and O′V are directionvectors after boundary passage and before boundary passage. Afterrefraction, the rays propagate back again to the point W on the arc QQ′.Thus, the optical path length L is calculated by examining the opticalpath from the point 0′ to the point W. The optical path length L isdetermined using, for example, Equation (4) below.

[Math. 4]

L=n ₁ |VW|+n ₂ |O′V|  (4)

The above calculation is performed on a plurality of rays to obtain theoptical path lengths of all the rays reaching the spherical crown. Aphase difference is obtained through the optical path difference fromthe optical path length, and a pattern for eliminating this phasedifference becomes an aberration correction pattern, that is, aberrationcorrection hologram data. In practice, since there are a plurality ofboundaries B1 of the refractive index, the rays may be refracted at eachboundary B1 of the refractive index and traced. In this case, Equation(4) is changed.

<Wavefront Propagation>

Using Fresnel diffraction, Fresnel-Kirchhoff diffraction, or the like,rays gradually propagate back from a position at which the rays condense(point O′) to the objective lens 12. In this case, the calculation isperformed by adding the boundary B1 of the refractive index to thepropagation. This method may be combined with the backward ray tracingdescribed above or electromagnetic field analysis to be described below.For example, reverse ray tracing may be performed on a portion otherthan the boundary B1 of the refractive index, and wavefront propagationmay be performed on a portion including the boundary B1 of therefractive index. Thus, it is possible to reduce a calculation load.

<Electromagnetic Field Analysis>

Using a Finite-Difference Time-Domain (FDTD) method or a RigorousCoupled-Wave Analysis (RCWA) method, analysis is performed from a pointO′ at which condensing is desired to the objective lens 12. In thiscase, the boundary B1 of the refractive index is added to boundaryconditions. This method may be combined with the above-describedbackward ray tracing or wavefront propagation.

Conversely propagation analysis using each method described above isapplicable even when there are two boundaries of the refractive index.FIG. 6 schematically illustrates a case where there are two refractiveindex boundaries B2 and B3. The boundary B2 is assumed to beperpendicular to the optical axis. Further, an angle between a plane Hperpendicular to the optical axis A2 and the boundary B3 is a. Therefractive index outside of the boundary B2 is n1, the refractive indexbetween the boundary B2 and the boundary B3 is n2, and the refractiveindex inside of the boundary B3 is n3.

FIGS. 7(a) and 7(b) are diagrams illustrating the wavefront obtainedusing the backpropagation analysis in such a case, and a phase isindicated by shading. FIG. 7 is represented using a technology calledphase folding. FIG. 7(a) illustrates a wavefront obtained when α=0°.Further, FIG. 7(b) illustrates a wavefront obtained when α=0.3°. In thecalculation of the wavefronts, n1=1, n2=1.33, and n3=1.38. In this case,n1 simulates air, n2 simulates water, n3 simulates a cell. It is assumedthat there is a cover glass at the boundary B2 between n1 and n2, but anaberration caused by the cover glass is corrected using a glasscorrection function of the objective lens 12.

A region sandwiched between the boundary B2 and the boundary B3 on theoptical axis (that is, a region sandwiched between the cover glass andthe cell) is filled with, for example, phosphate buffered saline. Adistance E1 between the boundary B2 and the boundary B3 on the opticalaxis is, for example, 30 μm. Further, a magnification of the objectivelens 12 is 40, a numerical aperture (NA) is 0.75, a distance E2 betweenthe point O and the boundary B3 on the optical axis is 300 μm, and adistance E3 between the point O and the point O′ is 80 μm. In the aboveconditions, a coma aberration and an astigmatism aberration are includedin the wavefront, in addition to the spherical aberration. Asillustrated in FIG. 7, a wavefront required for the irradiation light ispreferably obtained using the above-described backpropagation analysis.An aberration correction hologram is preferably obtained on the basis ofthe wavefront.

As a method of correcting the aberration caused by the surface shape ofthe biological sample B, for example, a method of combining a phasemodulator (mainly, a deformable mirror) with a wavefront measurementdevice (a Shack-Hartmann sensor) and embedding fluorescent beads with aknown size or shape at a particular position inside the biologicalsample B can be considered. In this method, excitation light andfluorescence from the fluorescent beads are both under an influence ofaberration. Therefore, the aberration is measured by measuring afluorescence intensity distribution using the wavefront measurementdevice. A hologram for correcting the aberration is presented to thephase modulator. In this case, the fluorescent beads are used asreference information.

However, in such a method, it is necessary for fluorescent beads to beembedded by surgery, and this method is not applicable in a case inwhich it is difficult to embed the fluorescent beads or a state of thebiological sample B being changed due to the embedded fluorescent beads.On the other hand, according to the method of this embodiment, it is notnecessary to embed the fluorescent beads.

Further, a method of obtaining an aberration correction hologram bytrial and error by presenting a plurality of holograms by which anaberration is assumed to be reduced to a spatial light modulator,scanning the irradiation light modulated by the spatial light modulator,and selecting a hologram by which luminance or resolution of an obtainedimage is improved may be considered.

However, such a method takes a long time to repeat an experiment anddesign. Further, the obtained aberration correction hologram is likelyto be an approximate solution, and accuracy is suppressed to be low. Onthe other hand, in the method of this embodiment, since the calculationof the aberration correction hologram data is all performed in thecomputer 43, it is possible to shorten the required time as comparedwith the method with trial and error of an operator. Further, in themethod of this embodiment, since the computer 43 generates theaberration correction hologram data on the basis of the information onthe surface shape acquired in the shape measurement unit 20, it ispossible to enhance accuracy of the aberration correction.

Further, a method of correcting an aberration by causing thefluorescence generated in the biological sample B to separate,condensing one of the separated fluorescences using a lens, and imaginga condensing image using the camera may also be considered (see PatentLiterature 1). The condensing image becomes a diffraction limited imagewhen there is no aberration caused by the surface shape of thebiological sample B, whereas the condensing image has a shape withdistortion when there is an aberration. Therefore, in this method, aplurality of aberration conditions according to the shape of a pluralityof condensing images are stored in advance, and an appropriate conditionis selected from among the conditions to perform aberration correction.

However, in such a method, the obtained aberration correction hologramis highly likely to be an approximate solution, and accuracy issuppressed to be low. In the method of this embodiment, since thecomputer 43 generates the aberration correction hologram data on thebasis of information on the surface shape acquired in the shapemeasurement unit 20, it is possible to enhance accuracy of aberrationcorrection.

The hologram presented to the first spatial light modulator 33 and thesecond spatial light modulator 36 may not be the aberration correctionhologram itself. For example, the hologram may be a hologram in whichanother hologram such as a hologram for controlling a condensing shapeor a focus position of the irradiation light L1 with which thebiological sample B is irradiated is superimposed on the aberrationcorrection hologram.

Second Embodiment

In the first embodiment, the correction of the aberration caused by thesurface shape of the biological sample B has been described, but anaberration may be caused by an internal structure directly under thesurface of the biological sample B. Hereinafter, such a phenomenon willbe described in detail.

Even when the biological sample B is irradiated with irradiation lightwith a certain light intensity, the fluorescence intensity to beobserved is different between a deep position and a shallow position inthe biological sample B. This is due to scattering and an aberration ofthe irradiation light or fluorescence caused by an internal structure ofthe biological sample B. One cause of occurrence of the scattering orthe aberration is a change in the optical path caused by a refractiveindex difference between organs constituting, macroscopically, astructure of a blood vessel or the like and, microscopically, a cell. Inparticular, at a deep position in the biological sample B, the opticalpath is changed due to the internal structure, and a condensing shape ofthe irradiation light is greatly changed. Thus, a decrease in thefluorescence intensity and the resolution which is observed and adecrease in an S/N ratio due to background noise occur.

Therefore, in this embodiment, an internal structure with a refractiveindex difference directly under the surface of the biological sample Bis measured, and a hologram including an aberration correction hologramfor correcting an aberration caused by the internal structure ispresented to the first spatial light modulator 33 and the second spatiallight modulator 36. Thus, a decrease in strength and resolution of thelight to be detected L2 and a decrease in the S/N ratio due tobackground noise can be suppressed.

FIGS. 8(a) to 8(d) are diagrams when the biological sample B is viewedfrom the side, and schematically illustrate a state in which irradiationlight is condensed while passing through an inner structure includingmedia Ba and Bb with different refractive indexes. FIGS. 8(a) and 8(b)illustrate a state of condensing of irradiation light L7 when noaberration is assumed, and the irradiation light L7 is condensed in ashallow position and a deep position in the biological sample B. Asillustrated in FIG. 8(a), when the irradiation light L7 is condensed inthe shallow position, the boundary of the refractive index present onthe optical path is relatively small. On the other hand, as illustratedin FIG. 8(b); when the irradiation light L7 is condensed in the deepposition, the boundary of the refractive index present on the opticalpath increases. Therefore, in such a case, the irradiation light isactually influenced by the internal structure of the biological sampleB, as illustrated in FIG. 8(c).

When the aberration correction is not performed as in FIG. 8(c), rays L7c and L7 d near a lens outer peripheral portion are refracted under aninfluence of the internal structure. On the other hand, since arefraction angle of rays near the optical axis A2 is small, the raysnear the optical axis A2 are less affected by the internal structure. Asa result, a focus position of the rays near the optical axis A2 isdifferent from a focus position of the rays L7 c and L7 d, and anaberration occurs. FIG. 8(d) illustrates a case in which such anaberration has been corrected. By correcting the wavefront of theirradiation light L7 in consideration of the refractive indexdistribution of the internal structure, the focus position of the raysnear the optical axis A2 matches the focus position of the rays L7 c andL7 d, and condensing of the irradiation light L7 can be achieved at highdensity.

FIG. 9 is a diagram illustrating a configuration of a microscopeapparatus 1B of this embodiment. A difference between the microscopeapparatus 1B and the microscope apparatus 1A of the first embodiment ispresence or absence of the shape measurement unit 20. That is, themicroscope apparatus 1B of this embodiment does not include the shapemeasurement unit 20 and, instead, the objective lens moving mechanism 13of the microscope unit 10 and the laser light source 31, the opticalscanner 35, and the detector 37 of the image acquisition unit 30constitute a shape acquisition unit that acquires information on theinternal structure directly under the surface of the biological sampleB. In this embodiment, the computer 43 acquires information on theinternal structure of the biological sample B on the basis of adetection signal S2 Obtained by moving the objective lens 12 in theoptical axis direction. The image acquisition unit 30 includes thedetector 37 and is optically coupled to the objective lens 12.

With the configuration of this embodiment, it is possible to acquireinformation on the internal structure directly under the surface of thebiological sample B. That is, when a suitable fluorescent material isused, information on the internal structure of the biological sample Bis included in the obtained fluorescence image. Therefore, a distancebetween the objective lens 12 and the biological sample B issequentially changed using the objective lens moving mechanism 13. Whenthe irradiation light (excitation light) L1 is condensed at a deepposition inside the biological sample B, fluorescence (includingautofluorescence) is emitted from a structure inside the biologicalsample B present on the optical path of the irradiation light(excitation light) L1. Accordingly, it is possible to recognize arefractive index distribution of the biological sample B by acquiringthe fluorescence intensity while setting a focus position shallower. Itis possible to reduce an influence of an aberration by presenting theaberration correction hologram taking the recognized refractive indexdistribution into account to the first spatial light modulator 33 andthe second spatial light modulator 36.

In this embodiment, the biological sample B may be a sample that emitsfluorescence from a specific portion due to a fluorescent dye, afluorescent protein, autofluorescence, or Second Harmonic Generation(SHG), or the like. Further, in the following description, the operationfor measurement of the internal structure as described above using theobjective lens moving mechanism 13, the laser light source 31, theoptical scanner 35, and the detector 37 is referred to as pre-scan.

In a laser scanning fluorescence microscope such as the microscopeapparatus 1B, scanning is performed by the optical scanner 35 (forexample, an XY galvanometer) in order to observe the fluorescent imagewithin a plane perpendicular to the optical axis. Information within aplurality of planes with different depths is obtained through a movementof the objective lens 12 or the biological sample table 11 in theoptical axis direction. Finally, three-dimensional information isconstructed by combining the information.

Further, the shape acquisition unit may measure the refractive indexdistribution using light sheet light used for fluorescence observationof a single photon, ultrasound, or the like, in place of the aboveconfiguration or together with the above configuration. Thus, it ispossible to recognize a rough structure before observation in advanceand preferably design an aberration correction hologram for correctingan aberration caused by the structure.

FIG. 10 is a flowchart illustrating an operation of the microscopeapparatus 1B described above, and an image acquisition method accordingto this embodiment. First, the biological sample B is placed on thebiological sample table 11. Then, a focus position is gradually movedfrom a shallow position in the biological sample B to a deep positionwhile moving the objective lens 12 in an optical axis direction. At thesame time, the light L5 is output from the laser light source 31 and theirradiation light L1 is condensed directly under the surface of thebiological sample B. Thus, in the internal structure directly under thesurface of the biological sample B, the light to be detected(fluorescence) L2 is generated. The light to be detected L2 is descannedby the optical scanner 35 and reflected by the dichroic mirror 34. Thedetector 37 detects the descanned light to be detected L2, and outputsthe detection signal S2.

In this case, the aberration correction is not performed by the firstspatial light modulator 33. Thus, a change in fluorescence in a depthdirection caused by the internal structure directly under the surface ofthe biological sample B is detected. This operation is performedrepeatedly while moving the optical axis of the irradiation light L1using the optical scanner 35. Accordingly, three-dimensional informationon the internal structure of the biological sample B is constructed. Arefractive index distribution is estimated from the constructedthree-dimensional information (shape acquisition step S21).

Thereafter, the hologram generation step S12, the light irradiation stepS13, the light detection step S14, and the image generation step S15 areperformed, similar to the first embodiment. In the observations at adeep position, since the irradiation light L1 and the light to bedetected L2 pass through a portion or all of the estimated refractiveindex distribution, an aberration correction hologram may be generatedso that an influence of the refractive index distribution on theirradiation light L1 and the light to be detected L2 passing through therefractive index distribution is reduced in the hologram generation stepS12. In this embodiment, the aberration correction hologram is designedon the basis of a wavefront obtained through backpropagation usinggeometrical optics, wave optics, electromagnetic field analysis, or thelike. The geometrical optics is, for example, backward ray tracing, thewave optics is, for example, Fresnel wavefront propagation or FresnelKirchhoff diffraction, and the electromagnetic field analysis is, forexample, FDTD or RCWA.

Although the aberration correction hologram is generated on the basis ofthe surface shape of the biological sample B in the first embodiment,and the aberration correction hologram is generated on the basis of thestructure with a refractive index distribution directly under thesurface of the biological sample B in the second embodiment, theaberration correction hologram may be generated on the basis of both ofthe surface shape of the biological sample B and the structure directlyunder the surface of the biological sample B.

Further, although the shape acquisition unit of this embodiment acquiresthe information on the structure directly under the surface of thebiological sample B using fluorescence obtained by radiating theirradiation light L1, the shape acquisition unit may measure therefractive index distribution directly under the surface of thebiological sample B using ultrasound or may measure the refractive indexdistribution directly under the surface using a phase difference or adifferential interference. Alternatively, the shape acquisition unit maychange an angle of the optical axis or change a refractive index of animmersion liquid and estimate the refractive index distribution or ascattering degree directly under the surface from reflection,transmission, or the like. Accordingly, it is possible to estimate,particularly, the refractive index distribution in the surface of thebiological sample B. For example, when the angle of the optical axis ischanged, the refractive index of the surface of the biological sample Bor the refractive index distribution inside the sample can be estimatedfrom a Brewster angle or a relationship between the angle and areflectance.

Effects obtained by the microscope apparatus 1B and the image acquiringmethod having the above configuration of this embodiment will bedescribed. In the microscope apparatus 1B and the image acquiring methodof this embodiment, the information on the internal structure directlyunder the surface of the biological sample B is acquired. On the basisof this information, the aberration correction hologram data forcorrecting the aberration is generated. Further, the irradiation lightL1 is modulated using the hologram based on the data. Thus, since theaberration caused by the internal structure directly under the surfaceof the biological sample B is preferably corrected, it is possible tosuppress a decrease in the condensing intensity of the irradiation lightL1 and spreading of the condensing shape inside the biological sample B.

Further, in this embodiment, the hologram based on the aberrationcorrection hologram data may be presented to the second spatial lightmodulator 36, and the light to be detected L2 generated in thebiological sample B may be modulated by the second spatial lightmodulator 36. Thus, since an influence of the aberration caused by theinternal structure directly under the surface of the biological sample Bon the light to be detected L2 is corrected, a clearer image of thebiological sample B can be obtained.

FIRST MODIFICATION EXAMPLE

FIG. 11 is a diagram illustrating an operation of a first modificationexample, and is a diagram when the surface of the biological sample B isviewed in the optical axis direction. In the second embodiment, theshape acquisition unit may divide a surface of the biological sample Binto a plurality of regions F1 to F4 in a grid pattern as illustrated inFIG. 11 and perform scanning in small areas in parallel in therespective regions F1 to F4 (in the shape acquisition step). In thiscase, an individual aberration correction hologram is generated for eachof the regions F1 to F4, and a hologram having effects of aberrationcorrection and multi-point generation is presented to the first spatiallight modulator 33.

FIGS. 12(a) and 12(b) are diagrams illustrating examples of a boundaryB1 at the outer side of the biological sample B. Since a distancebetween the objective lens 12 and the boundary B1 and an internalstructure directly under the surface of the biological sample B greatlychange when the irradiation light L1 scans a wide region as illustratedin FIG. 12(a), an aberration correction hologram may be switched midway.However, in this case, a loss of time is caused by a slow operation ofthe spatial light modulator. On the other hand, when a size of thescanning region is reduced through division into a plurality of regionsF1 to F4 as illustrated in FIG. 12(b), it is not necessary to switch theaberration correction hologram, and high-accuracy and high-speedscanning are realized.

SECOND MODIFICATION EXAMPLE

FIG. 13 is a diagram illustrating a configuration of a microscopeapparatus 1C according to a second modification example. In themicroscope apparatus 1C of this modification example, a shapemeasurement unit 20B constitutes the shape acquisition unit. The shapemeasurement unit 20B includes a light source 25, a dichroic mirror 26,an optical scanner 27, a detector 28, and a filter 29. The shapemeasurement unit 20B includes a detector 28 and is optically coupled toan objective lens 12.

The light source 25 outputs excitation light L8 with which a biologicalsample B is irradiated. In this modification example, the biologicalsample B may be a sample that emits fluorescence from a specific portiondue to a fluorescent dye, a fluorescent protein, autofluorescence, SHG,or the like, similar to the second embodiment. Then, the excitationlight L8 is light including a wavelength that excites the biologicalsample B. The excitation light L8 may be light with the same wavelengthas the light L5 output from the laser light source 31 or may be lightwith a wavelength different from that of the light L5 output from thelaser light source 31.

The dichroic mirror 26 transmits one of the excitation light L8 from thelight source 25 and fluorescence L9 from the microscope unit 10, andreflects the other. In the example illustrated in FIG. 13, the dichroicmirror 26 reflects the excitation light L8 and transmits thefluorescence L9.

The optical scanner 27 scans an irradiation position of the excitationlight L8 in the biological sample B by moving an optical axis of theexcitation light L8 within a plane perpendicular to the optical axis ofthe excitation light L8. The optical scanner 27 includes, for example, agalvanometer mirror, a resonant mirror, or a polygon mirror. Thefluorescence L9 from the biological sample B is detected through theoptical scanner 27. Thus, it is possible to match the optical axis ofthe excitation light L8 with the optical axis of the fluorescence L9.

The detector 28 detects a light intensity of the fluorescence L9 outputfrom the biological sample B via the objective lens 12, and outputs adetection signal S3. The detector 28 may be a point sensor such as aPMT, a photodiode, or an avalanche photodiode. Alternatively, thedetector 28 may be an area image sensor such as a CCD image sensor, aCMOS image sensor, a multi-anode PMT, or a photodiode array. A confocaleffect may be imparted due to a pinhole disposed at a preceding stagefrom the detector 28.

The filter 29 is disposed on an optical axis between the dichroic mirror26 and the detector 28. The filter 29 cuts a wavelength of theexcitation light L8 and a wavelength of fluorescence or the likeunnecessary for observation from the light that is input to the detector28.

When a distance between the objective lens 12 and the light source 25 islong, at least one 4f optical system may be provided on an optical axisof the excitation light L8 and the fluorescence L9. As an example, one4f optical system 53 is illustrated in FIG. 13. The 4f optical system 53is disposed on the optical axis between the optical scanner 27 and thebeam splitter 14.

In this modification, first, the biological sample B is placed on thebiological sample table 11. Then, a focus position is gradually movedfrom a shallow position in the biological sample B to a deep positionwhile moving the objective lens 12 in an optical axis direction. At thesame time, the light L8 is output from the laser light source 25 andfluorescence L9 from the internal structure directly under the surfaceof the biological sample B is detected in the detector 28. Thus, achange in the fluorescence in a depth direction caused by the internalstructure directly under the surface of the biological sample B isdetected. This operation is repeatedly performed while moving theoptical axis of the excitation light L8 using the optical scanner 27.Accordingly, three-dimensional information on the internal structure ofthe biological sample B is constructed. A refractive index distributionis estimated from the constructed three-dimensional information. Asubsequent operation is the same as in the second embodiment describedabove. When a wavelength of the excitation light L8 is included inwavelengths of the light L5 output from the laser light source 31, thefunctions of the light source 25 and the optical scanner 27 may berealized by the laser light source 31 and the optical scanner 35. Inthis case, the light source 25, the optical scanner 27, and the beamsplitter 26 are not necessary.

THIRD MODIFICATION EXAMPLE

In the second embodiment described above, pre-scan may be repeated aplurality of times, and in the second and subsequent pre-scans, anaberration of light L5 may be corrected by the first spatial lightmodulator 33 on the basis of a previous pre-scan result. Thus, when thebiological sample B is irradiated with the light L5, it is possible toreduce an influence of an aberration caused by the internal structure ofthe biological sample B, more accurately recognize a structure of thebiological sample B, and obtain a higher resolution image.

Specifically, in first pre-scan, the irradiation light L1 becomes planewaves since the aberration correction is not performed, but in secondpre-scan, a wavefront for correcting the aberration is applied to theirradiation light L1. The aberration correction hologram is designed onthe basis of a structure of the biological sample B obtained by thesecond and subsequent pre-scans. Alternatively, rough aberrationcorrection may be performed by the first pre-scan, and finer aberrationcorrection may be performed by the second and subsequent pre-scans.

FOURTH MODIFICATION EXAMPLE

FIG. 14 is a diagram illustrating a configuration of a microscopeapparatus 1D according to a fourth modification. In this modificationexample, an evanescent field is used for measurement of the surfaceshape of the biological sample B. Accordingly, shape recognition such asrecognition as to whether or not the biological sample B is grounded tothe biological sample table 11 is facilitated. In this modificationexample, the objective lens 12 and the biological sample table 11 (coverglass) in which total reflection occurs are used in order to generate anevanescent field.

In the microscope apparatus 1D of this modification example, the shapemeasurement unit 20C constitutes a shape acquisition unit. The shapemeasurement unit 20C includes a light source 25, a dichroic mirror 26, adetector 28, a filter 29, and a condensing lens 61. Configurations ofthe light source 25, the dichroic mirror 26, the detector 28, and thefilter 29 are the same as those in the third modification example. Theshape measurement unit 20C is optically coupled to the objective lens12.

The condensing lens 61 is provided when the excitation light L8 is asurface illumination, and is disposed on the optical axis between thedichroic minor 26 and the microscope unit 10. The condensing lens 61condenses the excitation light L8 on a rear focal plane of the objectivelens 12. When the excitation light L8 is a point illumination, thecondensing lens 61 is not necessary. In this case, for example, theexcitation light L8 which is plane waves may be input to a region inwhich total reflection of the objective lens 12 occurs and theexcitation light L8 may be scanned through parallel movement of theoptical scanner or the biological sample table 11. Further, a prism oran optical fiber may be used in place of the objective lens 12.

FIFTH MODIFICATION EXAMPLE

FIG. 15 is a diagram illustrating a configuration of a microscope unit10B and a shape measurement unit 20D of a microscope apparatus accordingto a fifth modification example. In this modification example,elasticity of the biological sample B is measured using ultrasound, anda structure of the biological sample B is obtained using an elasticitydifference between regions of the biological sample B. An aberrationcorrection hologram is generated on the basis of the obtained structure.The image acquisition unit 30 and the control unit 40 illustrated inFIGS. 1 and 9 are not illustrated in FIG. 15.

As illustrated in FIG. 15, the shape measurement unit 20D of thismodification example includes a pulse generation source 62 and areceiver 63 which is a detector. The pulse generation source 62generates a pulse signal S4 for generating ultrasound. The receiver 63receives the pulse signal S5 including information on an internalstructure of the biological sample B.

The microscope unit 10B includes a lens with a piezoelectric thin film64 in place of the objective lens 12 of the microscope unit 10illustrated in FIGS. 1 and 9. Other configurations are the same as thoseof the microscope unit 10. The lens with the piezoelectric thin film 64is disposed to face the biological sample B, converts a pulse signal S4to ultrasound using the piezoelectric thin film, irradiates thebiological sample B with the ultrasound, and converts the ultrasoundreflected in the biological sample B into a pulse signal S5. Although acommon lens with a piezoelectric thin film is used for the pulse signalS4 and the pulse signal S5 in this modification example, a lens with apiezoelectric thin film for the pulse signal S4 and a lens with apiezoelectric thin film for the pulse signal S5 may be providedindividually.

SIXTH MODIFICATION EXAMPLE

FIG. 16 is a diagram illustrating a configuration of a microscopeapparatus 1E according to a sixth modification example. In thismodification example, phase difference and differential interference areused for measurement of the surface shape of the biological sample B. Inthe microscope apparatus 1E of this modification example, the shapemeasurement unit 20E constitutes a shape acquisition unit. The shapemeasurement unit 20E includes a light source 21, a beam splitter 22, adetector 24, a differential interference (DIC) prism 66, and a polarizer67. The shape measurement unit 20E is optically coupled to the objectivelens 12. A configuration of the light source 21, the beam splitter 22,and the detector 24 is the same as that in the first embodiment.

The DIC prism 66 and the polarizer 67 are arranged side by side on theoptical path between the beam splitter 22 and the objective lens 12. TheDIC prism 66 causes the light L3 from the light source 21 to separate intwo, and also superposes return light L10 from the biological sample B.The polarizer 67 limits the polarization of the lights L3 and L10. Thelight L10 reaching the beam splitter 22 from the biological sample B viathe DIC prism 66 and the polarizer 67 is transmitted through the beamsplitter 22 and input to the detector 24.

The microscope apparatus and the image acquisition method according toan aspect of the present invention are not limited to theabove-described embodiments, and various modifications may be made. Forexample, although the aberration correction hologram is presented to thefirst spatial light modulator and the second spatial light modulator inthe above embodiment, the aberration correction hologram may bepresented to only the first spatial light modulator. In such a case, itis possible to suppress a decrease in the condensing intensity of theirradiation light inside the biological sample and spreading of thecondensing shape. Further, although the case in which the microscopeunit 10 is an inverted microscope has been described in the aboveembodiment, the microscope unit 10 may be an upright microscope.

The microscope apparatus according to the above embodiment is amicroscope apparatus that acquires an image of a biological sample, andincludes a biological sample table that supports the biological sample;an objective lens disposed to face the biological sample table; a lightirradiation unit that irradiates the biological sample with light viathe objective lens; a shape acquisition unit that acquires informationon at least one of a surface shape of the biological sample and astructure directly under the surface of the biological sample; ahologram generation unit that generates aberration correction hologramdata for correcting an aberration caused by the at least one on thebasis of the information acquired in the shape acquisition unit; aspatial light modulator to which a hologram based on the aberrationcorrection hologram data is presented and that modulates the light withwhich the biological sample is irradiated from the light irradiationunit; a photodetector that detects an intensity of light generated inthe biological sample; and an image generation unit that generates animage of the biological sample on the basis of an output from thephotodetector.

Further, an image acquisition method according to the above embodimentis a method of acquiring an image of a biological sample and includes ashape acquisition step of acquiring information on at least one of asurface shape of the biological sample supported by a biological sampletable facing an objective lens and a structure directly under thesurface of the biological sample; a hologram generation step ofgenerating aberration correction hologram data for correcting anaberration caused by the at least one on the basis of the informationacquired in the shape acquisition step; a light irradiation step ofpresenting a hologram based on the aberration correction hologram datato a spatial light modulator, modulating the light output from a lightirradiation unit using the spatial light modulator, and irradiating thebiological sample with light after modulation; a light detection step ofdetecting an intensity of light generated in the biological sample; andan image generation step of generating an image of the biological sampleon the basis of detection information in the light detection step.

Further, the microscope apparatus according to the above embodiment isan apparatus that acquires an image of a biological sample, and includesa biological sample table that supports the biological sample; anobjective lens disposed to face the biological sample table; a lightsource that outputs light with which the biological sample is irradiatedvia the objective lens; a shape acquisition unit that acquiresinformation on at least one of a surface shape of the biological sampleand a structure directly under the surface of the biological sample; ahologram generation unit that generates aberration correction hologramdata for correcting an aberration caused by the at least one on thebasis of the information acquired by the shape acquisition unit; aspatial light modulator to which a hologram based on the aberrationcorrection hologram data is presented and that modulates the lightoutput from the light source; a photodetector that detects an intensityof light generated in the biological sample and outputs a detectionsignal; and an image generation unit that generates an image of thebiological sample on the basis of the detection signal.

Further, the image acquiring method according to the above embodiment isa method of acquiring an image of a biological sample and includes ashape acquisition step of acquiring information on at least one of asurface shape of the biological sample supported by a biological sampletable facing an objective lens and a structure directly under thesurface of the biological sample; a hologram generation step ofgenerating aberration correction hologram data for correcting anaberration caused by the at least one on the basis of the informationacquired in the shape acquisition step; a light irradiation step ofpresenting a hologram based on the aberration correction hologram datato a spatial light modulator, modulating the light output from a lightsource using the spatial light modulator, and irradiating the biologicalsample with light after modulation; a light detection step of detectingan intensity of light generated in the biological sample and outputtinga detection signal; and an image generation step of generating an imageof the biological sample on the basis of a detection signal in the lightdetection step.

Further, the microscope apparatus may be configured to further include asecond spatial light modulator to which the hologram based on theaberration correction hologram data is presented and that modulates thelight generated in the biological sample. Further, in the imageacquiring method, the light detecting step may be configured to presentthe hologram based on the aberration correction hologram data to asecond spatial light modulator, modulate the light generated in thebiological sample using the second spatial light modulator, and detectan intensity of the light after modulation.

Accordingly, since an influence of an aberration caused by at least oneof the surface shape of the biological sample and the structure directlyunder the surface of the biological sample on the light generated in thebiological sample is corrected, a clear image of the biological samplecan be obtained.

Further, an incoherent light source that outputs incoherent light may beused in place of the laser light source 31. The incoherent light sourceincludes, for example, a super luminescent diode (SLD), a light emittingdiode (LED), an Amplified Spontaneous Emission (ASE) light source, or alamp-based light source.

INDUSTRIAL APPLICABILITY

One aspect of the present invention can be used as a microscopeapparatus and an image acquisition method capable of suppressing adecrease in condensing intensity of irradiation light inside abiological sample and spreading of a condensing shape.

REFERENCE SIGNS LIST

1A to 1E: microscope apparatus

10 and 10B: microscope unit

11: biological sample table

12: objective lens

13: objective lens moving mechanism

14: beam splitter

15: reflective mirror

20, 20B to 20E: shape measurement unit

21: coherent light source

22: beam splitter

23: reference light mirror

24: detector

30: image acquisition unit

31: laser light source

32: beam expander

33: first spatial light modulator

34: dichroic mirror

35: optical scanner

36: second spatial light modulator

37: detector

40: control unit

A2: optical axis

B: biological sample

L1: irradiation light

L2: light to be detected

L3: coherent light

L4: interference light

1-4. (canceled)
 5. A microscope comprising: a biological sample tableconfigured to support a biological sample; an objective lens disposed toface the biological sample table; a light source configured to outputlight with which the biological sample is irradiated via the objectivelens; a computer configured to generate aberration correction hologramdata for correcting an aberration caused by the at least one of asurface shape of the biological sample and a structure under the surfaceof the biological sample; a spatial light modulator configured tomodulate the light output from the light source based on the aberrationcorrection hologram data; and a photodetector configured to detect lightgenerated in the biological sample and outputting a detection signal. 6.The microscope according to claim 5, further comprising: an opticalscanner configured to scan an irradiation position of the modulatedlight in the biological sample.
 7. The microscope according, to claim 6,wherein the photodetector is configured to detect the light through theoptical scanner.
 8. The microscope according to claim 5, wherein thephotodetector is a point sensor.
 9. The microscope according to claim 5,wherein the computer is configured to generate the aberration correctionhologram data based on a refractive index distribution of the biologicalsample.
 10. The microscope according to claim 5, further comprising: ashape acquisition unit configured to acquire information on at least oneof the surface shape of the biological sample and the structure underthe surface of the biological sample.
 11. A microscopy methodcomprising: generating aberration correction hologram data forcorrecting an aberration caused by the at least one of a surface shapeof the biological sample supported by a biological sample table facingan objective lens and a structure under the surface of the biologicalsample; controlling a spatial light modulator based on the aberrationcorrection hologram data, modulating light output from a light sourceusing the spatial light modulator, and irradiating the biological samplewith the modulated light; and detecting light generated in thebiological sample and outputting a detection signal.
 12. The microscopymethod according to claim 11, further comprising: scanning anirradiation position of the modulated light in the biological sample.13. The microscopy method according to claim 12, wherein the detectingdetects the light through the light scanner.
 14. The microscopy methodaccording to claim 11, wherein the photodetector is a point sensor. 15.The microscopy method according to claim 11, wherein the generatinggenerates the aberration correction hologram data based on a refractiveindex distribution of the biological sample.
 16. The microscopy methodaccording to claim 11, further comprising: acquiring information on atleast one of the surface shape of the biological sample and thestructure under the surface of the biological sample.